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EVALUATION OF GREEN DOT’S LOCKE TRANSFORMATION PROJECT: 
SUPPLEMENTAL REPORT ON COHORT 2 STUDENT OUTCOMES 


Jordan Rickies, Jia Wang, and Joan Herman 
CRESST/University of California, Los Angeles 

Abstract 

With funding from the Bill and Melinda Gates Foundation, CRESST conducted a multi- 
year evaluation of a major school reform project at Alain Leroy Locke High School, 
historically one of California’s lowest performing secondary schools. Beginning in 2007, 
Locke High School transitioned into a set of smaller, Green Dot Charter High Schools, 
subsequently referred to as Green Dot Locke (GDL) in this supplemental report. This 
report extended the previous report, which tracked the first and second cohorts of 9*- 
graders who entered GDL in fall 2007 and 2008 respectively thru the 2010-11 school 
year, hy bringing the second cohort of students to graduation. The CRESST evaluation, 
employing a rigorous quasi-experimental design with propensity score matching, found 
statistically significant, positive effects for the GDL transformation including improved 
achievement, school persistence, graduation, and completion of college preparatory 
courses. 


Introduction 

With funding from the Bill and Melinda Gates Foundation, the National Center for 
Research on Evaluation, Standards, and Student Testing (CRESST) was charged with 
monitoring the progress of Green Dot Public Schools’ transformation of Alain Ixroy Locke 
High School. A CRESST report released in May 2012 (Herman et al., 2012) documents the 
results of this evaluation effort for students who attended Locke High School during the first 
four years of the transformation: 2007-08 through 2010-11 school years. In particular, the 
report focused on the first cohort of Green Dot Locke (GDL) students,^ as they entered 9* 
grade in 2007-08 and progressed through 12* grade in 2010-11. The report also tracked and 
analyzed the performance of the second cohort of GDL students, who started 9* grade in 
2008-09. Using a quasi-experimental design with propensity score matching to analyze a 
range of student performance outcomes, the study documented GDL’s statistically 
significant, positive effects on student achievement, school persistence, and completion of 
college preparatory courses. The report also documented that the positive effects were more 
prevalent for Cohort 2 than for Cohort 1 students. 


The Green Dot Locke Transformation Project began with two small, off-site schools in 2007-08 and 
completed in Fall 2008. The first cohort consisted of the students who started at these two small, off-site schools 
in 2007-08. 
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This report follows up the earlier study by examining how the second cohort of students 
fared through 12th grade and timely graduation. Because results for Cohort 2 students in 
years 1 through 3 were presented in the previous CRESST report, the main contribution of 
this supplemental report is the addition of effect estimates for year 4 outcomes. 

This report — using the same outcome measures and analytic methods described in the 
prior report — presents the findings from the follow-up analysis. The findings for Cohort 2 
students continue to reflect the positive and statistically significant results found for Cohort 1 
students. In particular, students who attended GDL, compared to demographically similar 
students attending demographically similar schools, were statistically more likely to stay at 
the same school for four years, take and pass key college preparatory courses, perform better 
on mathematics California Standards Tests (CST), pass the California High School Exit 
Exam (CAHSEE), and graduate within four years while meeting the course requirement to 
enter California state universities. While causal attribution is not straightforward given the 
study design, the study’s rigorous, quasi-experimental methods provide consistent, positive, 
and statistically significant evidence of Green Dot’s effectiveness in achieving its goals for 
improving public school education. 

In the following sections of this report, we review the evaluation methodology and 
present the findings. 

Evaluation Methodology 

Extending the findings of prior CRESST evaluation reports on the Eocke 
Transformation Project the current study addresses the following overarching question: 

Relative to their matched counterparts in LAUSD, how well did Cohort 2 students 
perform in terms of school persistence, attendance, course-taking and completion, as 
well as achievement on standardized tests in ELA and math during the first four years of 
high school? 

In this section, we review the available data, analytic strategy used to address this question, 
and the limitations of our evaluation methodology. 

Available Data 

This follow-up study combines new, student-level data acquired from EAUSD and 
Green Dot for the 2011-12 school year with available student- level data from the prior study. 
Together, these data incorporate students in Green Dot Public Schools and students in 
EAUSD local school districts 5, 7, 8, and T for the 2007-08, 2008-09, 2009-10, 2010-11, and 
2011-12 school years. In addition to demographic data, student outcome data included: 
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• School Persistence. Semester-by-semester indicator for whether the student stayed 
in the same school from the start of 9* grade through four years of high school. For 
GDL, “same school” was defined as enrolling in any of the GDL academies. 

• School Attendance. Annual school attendance rate. 

• Course-taking. Annual indicators, based on by-semester records, for the number of 
key college-ready courses a student enrolled in and whether they successfully 
completed those courses with a grade of C or better. For this report, we expanded 
the list of courses eligible for completion of key college-ready course requirements 
to better capture the extent to which students satisfy specific requirements within 
four years of high school. See the appendix for the list of courses included in our 
analysis. 

• Student Achievement. Multiple measures of student learning based on the 
California Standards Test (CST) scale scores and on the California High School 
Exit Examination in English language arts (EEA) and mathematics. Eor the 
CAHSEE, we examined performance for a student’s first time taking the test (both 
scale score and whether the student passed or not) and whether a student passed 
CAHSEE by the end of their fourth year in high school.^ 

• End-of-High School Measures. Indicators for whether students graduated from 
high school by the end of their fourth year (2011-12), and whether students 
graduated with completed A-G required courses, the coursework required for entry 
to California state universities. 

Analysis Strategy 

A quasi-experimental matching-based design was used to examine the transformation 
effects on GDE students. To estimate how GDE students would have performed on the 
various outcome measures in the absence of the GDE transformation, we matched GDE 
students to non-GDE students with similar 8* grade characteristics and academic 
performance. Cohort 2 GDE students were identified as attending GDE as a 9* grader in the 
fall semester of 2008-09 and attending one of six EAUSD feeder middle schools as an 8* 
grader in 2007-08. The pool of comparable non-GDE students were identified as attending 
one of three neighboring EAUSD high schools as a 9* grader in the fall semester of 2008-09 
and attending one of the six EAUSD feeder middle schools as an 8* grader in 2007-08.^’"^ 


^ Passing the CAHSEE was defined as scoring 350 or above and students who do not take the CAHSEE are 
considered not passing. This is a slight change in definition from previous GDL CRESST reports. This change 
was made to improve comparability of CAHSEE pass rates between GDL and EAUSD schools, but did not 
result in substantive differences in findings between this and previous reports. 

^ The three comparison high schools (Eremont, Jordan, or Washington Preparatory) for non-GDE students were 
identified as the EAUSD high schools that most students in the Locke feeder middle schools attended if they did 
not attend GDL. Eeeder middle schools were defined as schools having at least five students in the first GDL 9* 
grade cohort and at least ten students in the second cohort. The following six middle schools were identified as 
EAUSD feeder middle schools that Locke students graduated from; Bethune, Clay, Drew, Gompers, Harte, and 
Markham. In the previous CRESST reports, the analysis included students who did not attend one of the feeder 
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From the available pool of non-GDL students, control students were selected by 
matching them to GDL students on a number of 8* grade demographic and academic 
performance measures. A combination of exact matching and nearest-neighbor propensity 
score matching was implemented via the Matchit package for R (Ho, Imai, King, & Stuart, 
2009). Students were matched exactly by gender, ethnicity, parents’ education, poverty 
status, language classification, and 8* grade CSX math subtest taken. Within each exact 
match, a control student was identified for each treatment student based on nearest-neighbor 
propensity score matching (where the estimated propensity score was determined by the 
student’s 8* grade CSX scale scores for ELA and math as well as the student’s 8* grade 
attendance rate). By matching students based on their 8* grade characteristics, we can rule 
out concerns that differences in outcomes between the matched GDL and control students 
were due to these measured pre-existing differences between GDL and control students. As 
with most non-randomized designs, however, we cannot rule out concerns that group 
differences were due to unobserved student characteristics (e.g., motivation) rather than the 
GDL transformation. 

Since we examined a wide array of outcomes and not all students were measured on 
each outcome, we created different analytic matched samples based on the outcome(s) of 
interest. Lor the student persistence outcome, the Cohort 2 9* Grade Entry sample included 
students enrolled in high school as 9* graders in the 2008-09 fall semester and whose 8* 
grade CSX scores and demographics (from the 2007-08 school year) were available for 
matching. Xhe other student outcome measures — namely attendance, CSX, and course-taking 
and completion — were missing if a student was not enrolled in a given year. As a result, for 
the analysis of these end-of-year outcomes we defined the student sample of interest for each 
year as students who met the criteria for inclusion in the Cohort 2 persistence analysis and 
had course-taking information for both the fall and spring semesters, as well as ELA and 
math CSX scores in the same school year as the outcome(s) of interest. Lor example, the year 
3 achievement outcomes for Cohort 2 students were based on students for whom we had: (1) 
8* grade demographic and CSX data on both ELA and math in 2007-08; (2) course-taking 
information for the fall and spring semesters of the 2008-09, 2009-10, and 2010-11 school 

middle schools. This was primarily to ensure adequate sample sizes for the Cohort 1 analysis. Since this report 
focused on Cohort 2, which included more than twice as many students as Cohort 1, we restricted the analysis 
to students from the six feeder middle schools. This restriction did not substantively change the findings, but 
provides some assurances that the results are not biased by factors outside the local Locke neighborhoods and 
schools. 

In fall 2011, Green Dot took over the joint management of Jordan High with Mayor Villaraigosa's Partnership 
for Los Angeles Schools (PLAS). At the Jordan school site in 2011-12, both Green Dot and PLAS accepted 
enrollments independently; Green Dot accepted students in grades 9-11 and PLS accepted students in grades 9- 
12 . 
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years; and (3) CST scores for the 2008-09, 2009-10, and 2010-11 school years. The one 
exception is that the sample for year 4 outcomes (i.e., the 12* grade year) did not require 
available CST scores in that year. This approach raises some concerns (discussed in the 
limitations section), but offers a way to focus the analysis on annual effects. 

This approach provides us with five matched samples for Cohort 2: 

• 9* Grade Entry sample; 

• Year 1 (2008-09) end-of-year outcomes sample; 

• Year 2 (2009-10) end-of-year outcomes sample; 

• Year 3 (2010-11) end-of-year outcomes sample; and 

• Year 4 (201 1-12) end-of-year outcomes sample. 

Student characteristics for each of these matched samples are summarized in Table 1. As 
designed, the matching process produced treatment (i.e., GDL) and control (i.e., non-GDL) 
groups with identical student characteristic profiles and nearly identical average 8* grade 
CST and attendance records. For the Cohort 2 Year 4 sample, 248 of the 296 GDL students 
(84%) where matched to a non-GDL student. 

Limitations of Evaluation Methodology 

Like all studies, our analysis was constrained by available data and the conditions under 
which the GDL transformation was implemented. These overall constraints pose limitations 
in regards to the depth with which we could explore trends in academic outcomes and the 
extent to which one should interpret the effect estimates as causal. As we detailed these 
limitations in the earlier reports and as this is a supplemental report, the following is a brief 
discussion on the general limitations of the study’s design and the caveats to the causal 
interpretations of the results. 

Missing Data. Our analyses required the processing of student-level data from both 
Green Dot Public Schools and LAUSD. We did not have data on students who left GDL and 
LAUSD during the time period examined for this report, unless they subsequently attended 
another school represented in the provided data (i.e., another Green Dot school or an LAUSD 
school in local district 5, 7, 8, or T). As a result, we cannot examine outcomes for these 
students. Similarly, we did not have pre-high school data for students who entered GDL from 
outside the four local districts from which we received LAUSD data. Thus, our analyses 
examined students from specific local districts and who attended GDL at defined points in 
time; this did not capture all students exposed to the GDL transformation. 
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Data Comparability. Course taking is one of the most challenging outcomes to 
examine as the data were not aligned across GDL and LAUSD data sources (particularly in 
terms of course names/codes) and as we did not have access to summer school or intersession 
course-taking for LAUSD students. We attempted to address these two concerns by 
examining course enrollment and completion rates for broad subject areas rather than specific 
courses and we restricted our analysis to only fall and spring semesters. Given GDL’s heavy 
use of intercession courses for struggling students, this omission may underestimate the 
reported course-taking and pass rate effects for the GDL transformation. 

There could be some possible differences in how Green Dot and LAUSD track their 
students’ graduation and A-G completion data. In prior years we had to request additional or 
revised graduation and A-G data from both LAUSD and Green Dot after identifying 
inconsistencies in the data. Since we are comparing data from two separate systems, it is 
possible that some differences in outcomes are simply due to differences in data definitions 
and/or quality across the two systems. 

Casual Effects. In the absence of random assignment, observed differences between 
GDL and non-GDL students could also be due to pre-existing differences between the 
students (e.g., ability and motivation) rather than exposure to the transformation alone. 
However, by matching GDL students to non-GDL students with similar 8* grade 
characteristics and test performance observed in the data, we were able to rule out these 
measured factors as causing outcome differences between matched GDL and non-GDL 
students. This provided some credibility to claims that the observed differences were due to 
the GDL transformation. We were not, however, able to rule out the possibility that some 
unavailable pre-existing factors (absent from the available data and the matching process) 
explained the observed group differences instead of the transformation. 
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Table 1 

Comparison of Matched Non-Green Dot Locke & Green Dot Locke Students by 8th Grade Characteristics 
(Cohort 2) 



9th Grade 
Entry 

Year 1 

Year 2 

Year 3 

Year 4 

Characteristics 

Non- 

GD 

GD 

Non- 

GD 

GD 

Non- 

GD 

GD 

Non- 

GD 

GD 

Non- 

GD 

GD 

Number of students in cohort 

1,460 

578 

1,265 

524 

991 

428 

745 

338 

681 

296 

Number of matched students 

512 

512 

444 

444 

362 

362 

282 

282 

248 

248 

% Female 

52% 

52% 

51% 

51% 

49% 

49% 

51% 

51% 

53% 

53% 

Race/Ethnicity (%): 

Black / Afr. Am. 

25% 

25% 

22% 

22% 

19% 

19% 

17% 

17% 

16% 

16% 

Latino / Hispanic 

75% 

75% 

78% 

78% 

81% 

81% 

83% 

83% 

84% 

84% 

Parent's education (%): 

High school graduate 

22% 

23% 

22% 

24% 

22% 

25% 

24% 

25% 

23% 

24% 

Less than high school 

31% 

31% 

30% 

30% 

32% 

32% 

34% 

34% 

37% 

37% 

Unknown 

44% 

44% 

43% 

43% 

40% 

40% 

37% 

37% 

35% 

35% 

% Free/reduced lunch 

88% 

88% 

88% 

88% 

89% 

89% 

91% 

91% 

92% 

92% 

Language classification (%): 

English Only or IFEP 

31% 

31% 

27% 

27% 

24% 

24% 

22% 

22% 

21% 

21% 

RFEP 

34% 

34% 

35% 

35% 

38% 

38% 

40% 

40% 

44% 

44% 

English Learner 

35% 

35% 

38% 

38% 

38% 

38% 

38% 

38% 

35% 

35% 

% Students w/ disabilities 

8% 

8% 

7% 

7% 

6% 

6% 

5% 

5% 

3% 

3% 

Mean attendance rate 

94% 

94% 

95% 

95% 

95% 

95% 

96% 

95% 

96% 

96% 

Mean ELA CST scale score 

294 

295 

293 

294 

297 

297 

300 

302 

304 

305 

Took Algebra 1 CST: 

% Took test 

52% 

52% 

54% 

54% 

55% 

55% 

59% 

59% 

58% 

58% 

Mean scale score 

279 

282 

281 

284 

283 

286 

284 

288 

288 

293 

Took General Math CST: 

% Took test 

48% 

48% 

46% 

46% 

45% 

45% 

41% 

41% 

42% 

42% 

Mean scale score 

270 

276 

272 

271 

275 

273 

278 

275 

284 

276 


Effects of GDL on Student Performance 

Despite limitations described above, the current evaluation provides consistent and 
important evidence that GDL had positive effects on Cohort 2 students on a range of student 
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outcomes, as we found previously. In this section we present specific estimates of GDL 
effects for the Cohort 2 matched samples. We present two types of effect estimates: (1) mean 
(raw) differences between the matched GDL and non-GDL groups; and (2) regression 
adjusted differences based on linear regression models that adjust for potentially lingering 8* 
grade CST and/or attendance rate differences between the matched GDL and non-GDL 
students. In just about every instance the two types of estimates provide substantively similar 
findings, which partially validate the matching approach.^ 

School Persistence 

Findings from the previous CRESST report indicated that GDL students in Cohort 1 
were no more or less likely to stay in the same school during their first four years of high 
school compared to similar non-GDL students. For both the matched GDL and non-GDL 
students in Cohort 1, only 49% of the students who started 9th grade in the fall of 2007-08 
remained in the same school through the spring of 2010-11. For Cohort 2, however, the prior 
report suggested that GDF students were more likely to stay in the same school through the 
first three years of high school. The current report sought to determine whether this positive 
trend for Cohort 2 continued through the fourth year of high school. 

Results based on four years of Cohort 2 student data confirm the prior findings that 
GDF had a positive effect on school persistence. The percent of matched Cohort 2 students 
who stayed in the same school over the four year period (2008-09 to 2011-12) was about 
eight percentage points higher for GDF students compared to the matched non-GDF students 
(59% vs. 51%). Semester-by-semester results are presented in Table 2, while the persistence 
trend for Cohort 2 is displayed in Figure 1 . The average persistence rate difference between 
GDF and non-GDF students increased over the first two years of high school, then remained 
relatively steady during the last two years. This suggests that GDF’s effect on school 
persistence was particularly important during those critical first two years of high school. 

The adjusted effect estimates are summarized in Figure 2 with their approximate 95% 
confidence intervals. Estimates with a confidence interval that does not intersect with the 
zero line are considered statistically significant. For Cohort 2, the difference between the 
GDF and non-GDF matched students was positive and statistically significant in all 
semesters during the four year period. It is important to note, however, that Cohort 2’s 


^ We also present p-values in the tables and approximate 95% confidence intervals in the figures for 
interpretation of statistically significant differences. These statistical significance indicators are based on 
ordinary least squares (OLS) regression model estimates. In cases where the outcome is a dichotomous indicator 
(e.g., graduate or not), we also tested for statistical significance using a more statistically appropriate logistic 
regression model. We note instances where the two significance tests result in different conclusions. 
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overall persistence rate, while higher than for Cohort 1, remains low; 51% for non-GDL 
students and 59% for GDL students. Additional research is needed to understand what 
specific factors produced the positive findings and how those factors can be mobilized to 
raise persistence rates for all students. 

Table 2 


Estimated Effect of Green Dot Locke on Proportion of Students Staying in Same School, by 
Semester (Cohort 2 Matched 9th Grade Entry Sample) 



Control group 

Green Dot 
group 

Raw difference 

Adjusted 

difference* 

Year/Semester 

N 

Mean 

N 

Mean 

Estimate (p-value) 

Estimate (p-value) 

Year 1 Eall 

512 

1.00 

512 

1.00 

- 

- 

- 

- 

Year 1 Spring 

512 

0.91 

512 

0.95 

0.04 

(0.016) 

0.04 

(0.016) 

Year 2 Eall 

512 

0.78 

512 

0.84 

0.05 

(0.026) 

0.05 

(0.027) 

Year 2 Spring 

512 

0.72 

512 

0.80 

0.08 

(0.003) 

0.08 

(0.004) 

Year 3 Eall 

512 

0.63 

512 

0.73 

0.10 

(0.001) 

0.09 

(0.001) 

Year 3 Spring 

512 

0.60 

512 

0.67 

0.07 

(0.023) 

0.07 

(0.025) 

Year 4 Eall 

512 

0.54 

512 

0.61 

0.08 

(0.014) 

0.07 

(0.015) 

Year 4 Spring 

512 

0.51 

512 

0.59 

0.08 

(0.010) 

0.08 

(0.011) 


Note. Results are for students in the matched sample for a given year. * The adjusted difference 
controls for a student's 8th grade ELA CST scale score. 
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Figure 1. Proportion of students in matched 9th Grade Entry Sample who stayed in the 
same school by semester (Cohort 2) 


Year 1 
Spring 

Year 2 
Fall 

Year 2 
Spring 

Year 3 
Fall 


Year 3 
Spring 

Year 4 
Fall 

Year 4 
Spring 

-0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 

Estimated Effect on Persistence Rate 

Figure 2. Summary of estimated Green Dot effects on proportion of students 
staying in same school, by semester (Cohort 2 matched 9th Grade Entry 
Sample). Reported point estimates (diamonds) and approximate 95% confidence 
intervals (horizontal bars) are based on the adjusted regression probability 
estimates. 

School Attendance 

Findings from the previous CRESST report indicated that GDL and non-GDL school 
attendance rates were similar during the first three years of high school for Cohort 1 and 
Cohort 2 matched samples. The prior report did find a statistically significant positive GDL 
effect in the fourth year for Cohort 1, however. For this report we sought to determine 
whether this positive year 4 effect held for Cohort 2 students. 

The school attendance rate results for Cohort 2 (see Table 3) support findings from the 
previous report. During the first three years of high school, GDL and non-GDL students had 
similar attendance rates, on average. In the fourth year, the attendance rate for GDL students 
was slightly higher than for matched non-GDL students (94% vs. 93%). While small, this 
positive difference is statistically significant (see Figure 3). Additional research is needed to 
understand why fourth-year students have slightly higher attendance rates in GDL. 
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Table 3 


Estimated Effect of Green Dot Locke on School Attendance Rate, by Year (Cohort 2 Matched Samples) 


Year 

Control group 

Green Dot group 

Raw difference 

Adjusted difference* 

N 

Mean 

N 

Mean 

Estimate 

(p-value) 

Estimate 

(p-value) 

Year 1 

444 

0.93 

444 

0.92 

0.00 

(0.732) 

0.00 

(0.660) 

Year 2 

356 

0.93 

362 

0.93 

0.00 

(0.970) 

0.00 

(0.871) 

Years 

282 

0.94 

282 

0.94 

0.00 

(0.988) 

0.00 

(0.939) 

Year 4 

248 

0.93 

248 

0.94 

0.02 

(0.021) 

0.01 

(0.024) 


Note. Results are for students in the matched sample for a given year. *The adjusted difference controls 
for a student's 8th grade attendance rate and ELA CST scale score. 


Year 1 
Year 2 
Year 3 
Year 4 


Figure 3. Summary of estimated Green Dot effects on school attendance rate, by year (Cohort 2 matched 
samples). Reported point estimates (diamonds) and approximate 95% confidence intervals (horizontal bars) are 
based on the adjusted regression probability estimates. 

Course- Taking and Completion 

To gauge whether the Green Dot transformation increased exposure and successful 
completion of important high school courses, we examined the frequency with which 
students took and passed some of the key college-ready courses within the English, math, 
science, and social science subject areas. Findings from the previous CRESST report 
indicated that GDE students were as likely, or more likely, to take and pass key courses that 



Estimated Effect on Attendance Rate 
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meet the UC/CSU A-G requirements.^ For this report, we focused on the number of A-G 
eligible courses students in Cohort 2 took and passed with a C or better by the end of their 
fourth year of high school. The results presented below are based on Cohort 2 Year 4 
matched sample’. 

Results for the Year 4 sample (see Table 4) support the positive GDL findings from the 
previous report. Overall, GDL students were more likely to take and pass the number of 
courses recommended for college eligibility. Over the four years of high school, 93% of 
GDL students in the Year 4 sample and only 64% of matched non-GDL students took all the 

o 

key academic courses per our definition. The significant difference in course enrollment and 
completion between GDL and non-GDL students was particularly true when it came to the 
courses students likely took in their junior and senior years. For example, among Cohort 2 
students in the matched Year 4 sample, 47% of the GDL students took and passed 4 + English 
courses compared to 35% of non-GDL students, and 59% of the GDL students took and 
passed 3-i- math courses compared to 43% of non-GDL students. GDL students were also 
much more likely to take 3-i- science courses (88% vs. 38%), with a third science course not 
required for college eligibility but recommended by some higher education institutions. For 
the Year 4 sample, the estimated GDL effect on course enrollments are displayed in Figure 4, 
and the estimated effect on course pass rates are displayed in Figure 5. 

Table 4 

Estimated Effect of Green Dot Locke on Course-Taking and Completion Rates, by Subject (Cohort 2 Year 4 

Matched Sample) 


Control group Green Dot group Raw difference Adjusted difference* 


Year 

N 

Mean 

N 

Mean 

Estimate (p-value) 

Estimate 

(p-value) 

English 









Took 2+ Courses 

248 

1.00 

248 

1.00 

0.00 

(1.000) 

0.00 

(0.985) 

Passed 2+ Courses 

248 

0.79 

248 

0.83 

0.04 

(0.257) 

0.04 

(0.267) 

Took 3+ Courses 

248 

0.99 

248 

0.99 

0.00 

(1.000) 

0.00 

(0.986) 

Passed 3+ Courses 

248 

0.58 

248 

0.65 

0.07 

(0.117) 

0.07 

(0.120) 

Took 4+ Courses 

248 

0.89 

248 

0.97 

0.08 

(0.000) 

0.08 

(0.000) 


® Eor freshmen admission to UC and CSU system, students are required to have four years of English, three 
years of math, two years of social science, two years of science, one year of visual and performing arts, and two 
years of foreign language. Please refer to http://www.cde.ca.gov/ci/gs/hs/hsgrtable.asp for more details. 

^ We also examined course-taking for the matched 9* Grade Sample, which includes students who left school 
during the four year period, and found similar results. 

* Our complete list of key academic courses includes four or more English courses, three or more math courses, 
two or more science courses, and two or more history/social science courses. 
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Passed 4+ Courses 

248 

0.35 

248 

0.47 

0.12 

(0.006) 

0.12 

(0.005) 

Math 

Took 2+ Courses 

248 

1.00 

248 

0.99 

-0.01 

(0.157) 

-0.01 

(0.147) 

Passed 2+ Courses 

248 

0.68 

248 

0.74 

0.06 

(0.139) 

0.06 

(0.139) 

Took 3+ Courses 

248 

0.85 

248 

0.95 

0.10 

(0.000) 

0.10 

(0.000) 

Passed 3+ Courses 

248 

0.43 

248 

0.59 

0.16 

(0.000) 

0.16 

(0.000) 

Science 

Took 2+ Courses 

248 

0.99 

248 

1.00 

0.01 

(0.157) 

0.01 

(0.161) 

Passed 2+ Courses 

248 

0.53 

248 

0.75 

0.22 

(0.000) 

0.21 

(0.000) 

Took 3+ Courses 

248 

0.38 

248 

0.88 

0.50 

(0.000) 

0.49 

(0.000) 

Passed 3+ Courses 

248 

0.14 

248 

0.51 

0.37 

(0.000) 

0.36 

(0.000) 

History/Social Science 

Took 2+ Courses 

248 

0.87 

248 

1.00 

0.13 

(0.000) 

0.13 

(0.000)" 

Passed 2+ Courses 

248 

0.49 

248 

0.69 

0.21 

(0.000) 

0.20 

(0.000) 

Key Academic 
Courses 

Took All 

248 

0.64 

248 

0.93 

0.29 

(0.000) 

0.29 

(0.000) 

Passed All 

248 

0.17 

248 

0.40 

0.23 

(0.000) 

0.23 

(0.000) 


Note. Course completion/pass defined as passing with a C or better. 

* The adjusted difference controls for a student's 8th grade ELA CST scale score, 
not statistically significant at p < 0.05 level when tested with a logistic regression model. 

^ Key academic courses defined as 4 or more English courses, 3 or more math courses, 2 or more science 
courses, and 2 or more history/social science courses. 
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Figure 4. Summary of estimated Green Dot effects on course enrollment rates, by subject (Cohort 2 Year 4 
matched sample). Reported point estimates (diamonds) and approximate 95% confidence intervals (horizontal 
bars) are based on the adjusted regression probability estimates. Notes: effect on taking 2+ social science 
courses is not statistically significant at p < 0.05 level when tested with a logistic regression model. Key 
academic courses defined as 4 or more English courses, 3 or more math courses, 2 or more science courses, and 
2 or more history/social science courses. 
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Estimated Effect on Course Completion Rates 

Figure 5. Summary of estimated Green Dot effects on course completion rates, by subject (Cohort 2 Year 4 
matched sample). Completion rates defined as passing with a C or better. Reported point estimates (diamonds) 
and approximate 95% confidence intervals (horizontal bars) are based on the adjusted regression probability 
estimates. Notes: Key academic courses defined as 4 or more English courses, 3 or more math courses, 2 or 
more science courses, and 2 or more history/social science courses. 


Student Achievement: California Standards Test 

Since students only take the CST in grades 9-11, CST performance results for Cohort 2 
were included in the previous CRESST report. For completeness in this supplemental report, 
however, we replicated the CST analysis for Cohort 2. As discussed in the prior report, GDL 
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students performed similarly on the ELA CSX compared to the matched non-GDL students 
and outperformed non-GDL students on most of the math CSTs. 

Cohort 2 results from the CSX analysis are presented in Xable 5 by year and test for the 
matched samples. For the math tests, only those tests that represented the two main math 
courses in each grade are reported. Estimated effects and their corresponding approximate 
95% confidence intervals are displayed in Figure 6. 

Table 5 


Estimated Effect of Green Dot Locke on CST Scale Scores, by Year and Test (Cohort 2 Matched 
Samples) 



Control group 

Green Dot group 

Raw difference 

Adjusted difference* 

Year 

N 

Mean 

N 

Mean 

Estimate (p-value) 

Estimate 

(p-value) 

Year 1 

ELA 

444 

301 

444 

305 

3.92 

(0.184) 

3.42 

(0.073) 

Algebra 1 

339 

267 

375 

267 

0.61 

(0.817) 

-0.30 

(0.900) 

Geometry 

90 

270 

69 

295 

24.57 

(0.000) 

17.48 

(0.000) 

Year 2 

ELA 

362 

296 

362 

300 

3.71 

(0.268) 

3.82 

(0.098) 

Geometry 

203 

252 

205 

255 

3.52 

(0.251) 

3.12 

(0.259) 

Algebra 2 

116 

257 

113 

270 

12.33 

(0.039) 

10.99 

(0.022) 

Year 3 

ELA 

282 

298 

282 

305 

6.59 

(0.127) 

4.71 

(0.107) 

Algebra 2 

128 

245 

172 

254 

9.22 

(0.021) 

11.01 

(0.003) 

Sum. Math 

66 

244 

73 

273 

28.84 

(0.000) 

26.80 

(0.000) 


Note. Results are for students in the matched sample for a given year. * The adjusted difference 
controls for a student's 8* grade ELA CST scale score for ELA outcomes and 8* grade ELA and 
Math CST scale scores for Math outcomes. 
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Year 1 
ELA 


Year 2 
ELA 


Years 

ELA 


Year 1 
Algebra 1 

Year 1 
Geometry 

Year 2 
Geometry 

Year 2 
Algebra 2 


Year 3 
Algebra 2 

Year 3 
Summative 


Figure 6. Summary of estimated Green Dot effects on school attendance rate, by year and test (Cohort 2 
matched samples). Reported point estimates (diamonds) and approximate 95% confidence intervals (horizontal 
bars) are based on the adjusted regression probability estimates. 
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Student Achievement: California High School Exit Exam 

In the previous CRESST report, we found that Cohort 1 GDL students were no more, 
or less, likely to pass the California High School Exit Exam (CAHSEE) compared to 
matched non-GDE students. This null finding occurred for first time test-takers and for the 
overall pass rate after four years of high school. Eor Cohort 2 students, however, results from 
the previous report indicated GDE students outperformed their matched non-GDE 
counterparts on their first CAHSEE attempt and were more likely to have passed CAHSEE 
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by the end of their third year. For this report, we examined whether the positive Cohort 2 
findings held through four years of high school. 

Cohort 2 results for first attempt CAHSEE performance and pass rate by the end of four 
years are presented in Table 6. To improve comparability between GDE and non-GDE 
students, as well as retain as many students as possible in the analysis, we made some minor 
definition changes to our analysis of CAHSEE performance in this report.^ These minor 
modifications to our analytic approach did not substantively change the findings from the 
previous report. 


Table 6 

Estimated Effect of Green Dot Locke on CAHSEE Performance, by Attempt and Test (Cohort 2 Matched 
Samples) 


Year 

Control group 

Green Dot group 

Raw difference 

Adjusted difference* 

N 

Mean 

N 

Mean 

Estimate (p-value) 

Estimate 

(p-value) 

1st Attempt (Year 2 

Sample): 








ELA Scale Score 

348 

353 

357 

356 

2.67 

(0.244) 

2.94 

(0.064) 

Math Scale Score 

346 

354 

353 

362 

8.38 

(0.000) 

8.47 

(0.000) 

% Passed ELA 

362 

0.51 

362 

0.59 

0.07 

(0.044) 

0.08 

(0.012) 

% Passed Math 

362 

0.51 

362 

0.59 

0.08 

(0.025) 

0.08 

(0.016) 

Ever Passed (Year 4 

Sample): 








% Passed ELA 

248 

0.90 

248 

0.94 

0.04 

(0.073) 

0.04 

(0.070) 

% Passed Math 

248 

0.91 

248 

0.96 

0.05 

(0.033) 

0.05 

(0.025)" 

Ever Passed (9* Grade Entry Sample): 







% Passed ELA 

512 

0.61 

512 

0.70 

0.09 

(0.003) 

0.08 

(0.002) 

% Passed Math 

512 

0.61 

512 

0.71 

0.10 

(0.001) 

0.08 

(0.005) 


Note. Results are for students in the matched sample for a given year. Pass rates were determined based on a 
student scoring at or above 350 on a given test. *The adjusted difference controls for a student's 8* grade ELA 
CST scale score for ELA outcomes and 8* grade ELA and Math CST scale scores for Math outcomes. " Not 
statistically significant at p < 0.05 level when tested with a logistic regression model. 


^ Eor the previous report, we limited the CAHSEE analysis to include only students who took CAHSEE and 
limited the analysis of first-time CAHSEE test takers only to 10* graders. Eor this report, we consider students 
who did not take CAHSEE as not passing CAHSEE and the analysis of first-time test takers included students 
who took CAHSEE in any year as long as it was their first recorded CAHSEE score. Eor example, students who 
were retained in 9* grade for their second year of high school may not have taken CAHSEE in their second 
year, but did take CAHSEE for the first time in their third year (when they were officially promoted to 10* 
grade). 
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For both ELA and math, GDL Cohort 2 students had higher average scale scores than 
the matched non-GDL students on the first attempt, although the difference was only 
statistically significant for math. GDL students were also more likely to pass CAHSEE on 
their first attempt. Additionally, for students who remained in the same schools for four years 
(Year 4 Sample), a higher proportion of GDL students passed the ELA test (94% vs. 90%) 
and the math test (96% vs. 91%) by the end of their fourth year. The pass rate difference was 
only statistically different for math, however (see Eigure 7). While almost all of the Year 4 
sample students passed CAHSEE after one or multiple attempts, it is important to note that of 
all the students in the matched sample who started in 9* grade (9* Grade Entry Sample) less 
than three-fourths passed CAHSEE. Eor GDL students in the 9* Grade Entry Sample, 70% 
and 71% passed ELA and math, respectively, within four years, while 61% of non-GDL 
students in the 9* Grade Entry Sample passed ELA and math. 


1 st Attempt 
Pass ELA 


1 st Attempt 
Pass Math 


By Year 4 
Pass ELA 


By Year 4 
Pass Math 


Figure 7. Summary of estimated Green Dot effects on CAHSEE pass rates, by attempt and test (Cohort 2 
matched samples). Reported point estimates (diamonds) and approximate 95% confidence intervals (horizontal 
bars) are based on the adjusted regression probability estimates. Note; effect on passing math portion of 
CAHSEE by end of year 4 is not statistically significant at p < 0.05 level when tested with a logistic regression 
model. 

End-of-High School Measures 

In the previous CRESST report, we examined graduation rates and A-G completion for 
Cohort 1 students. Those results indicated that GDL students had much higher graduation 
and A-G completion rates than matched non-GDL students. Eor this report, we examined 
whether similar high school completion findings occurred with Cohort 2 students. 



Estimated Effect on CAHSEE Pass Rate 
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Cohort 2 graduation and A-G completion rates are reported in Table 7. We report 
outcomes for two different Cohort 2 matched samples: the 9* Grade Entry Sample that 
includes students who started 9* grade but may have left school prior to the end of 12* 
grade, and the Year 4 Sample that only includes students who stayed in the same school and 
had valid data for all four years of high school. Any student for whom we did not have 
graduation or A-G completion data was coded as a non-graduate and/or non-A-G 
completer. 

While only about half of the Cohort 2 students who entered GDL as a 9* grader 
graduated within four years, the findings indicate that GDL students were more likely to 
graduate than the matched non-GDL students. Among all matched students who started in 
GDL in 9* grade, 54% graduated within four years while only 40% of the matched students 
who started in a non-GDL school graduated. If we only look at students who remained in the 
same school for four years, 87% of GDL students and 70% of matched non-GDL students 
graduated. These graduation rate differences are sizable and statistically significant. 

Similarly, GDL students were more likely to graduate with their A-G course 
requirements. Among all matched Cohort 2 students who entered GDL as a 9* grader, 26% 
graduated and completed their A-G requirements compared to only 16% of matched non- 
GDL students. Of students who stayed in the same school over the four year period (Year 4 
sample), 44% of GDL students graduated and completed their A-G requirements compared 
to 27% of the matched non-GDL students. This indicates that about half of Cohort 2 GDL 
graduates met the A-G requirements while just over a third of matched non-GDL graduates 
met the A-G requirements. Lor both the 9* Grade Entry and Year 4 samples, differences in 
the graduation and graduation with A-G rates were statistically significant (see Ligure 8). 


*** It is important to note that this coding approach may incorrectly classify some students as non-graduates if 
they moved to — and subsequently graduated from — a school for which we did not receive data. We only 
received data for students in Green Dot Public Schools and LAUSD local districts 5, 7, 8 and T. 
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Table 7 

Estimated Effect of Green Dot Locke on Graduation Rates, by Cohort (Cohort 2 Matched Samples) 

Control group Green Dot group Raw difference Adjusted difference* 


Year 

N 

Mean 

N 

Mean 

Estimate (p-value) 

Estimate 

(p-value) 

9* Grade Entry Sample 

% Graduated 

512 

0.40 

512 

0.54 

0.15 

(0.000) 

0.14 

(0.000) 

% Graduated w/ A-G 

512 

0.16 

512 

0.26 

0.10 

(0.000) 

0.09 

(0.000) 

Year 4 Sample 

% Graduated 

248 

0.70 

248 

0.87 

0.17 

(0.000) 

0.18 

(0.000) 

% Graduated w/ A-G 

248 

0.27 

248 

0.44 

0.17 

(0.000) 

0.17 

(0.000) 


Note. Results are for students in the matched sample for a given year. *The adjusted difference controls for 
a student's 8th grade ELA CST scale score. 



Figure 8. Summary of estimated Green Dot effects on graduation rates (Cohort 2 matched samples). Reported 
point estimates (diamonds) and approximate 95% confidence intervals (horizontal bars) are based on the 
adjusted regression probability estimates. 


Conclusion 

The results presented in this report, and in previous CRESST reports, document 
consistent and important evidence that students who attended Locke during the Green Dot 
transformation had a more stable and higher achieving high school experience than similar 
students who attended a neighboring high school. In particular, when comparing Cohort 2 
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GDL students to matched non-GDL students, the following positive, statistically significant, 
findings emerged: 

• School Persistence: GDL students were more likely to stay in the same school over 
four years of high school (59% vs. 51%); 

• Course-taking and Course-completion: GDL students were more likely to take 
key academic college-prep (A-G) courses during the first four years of high school 
(93% vs. 64%); 

• CAHSEE: GDL students were more likely to pass CAHSEE on their first attempt 
(59% vs. 51% for EEA and Math); for students who remained in the same schools 
for four years. GDE students were more likely to pass CAHSEE math test; 

• CSX: GDE students scored better, on average, on high school CSX mathematics 
tests; 

• Graduation: GDE students were more likely to graduate from high school within 
four years (54% vs. 40%); 

• Graduation and A-G completion: GDE fourth-year students were more likely to 
graduate with successful completion of the A-G college eligibility course 
requirements (44% vs. 27%). 

While these findings indicate GDE students, on average, had a more positive academic 
high school experience than their non-GDE counterparts, one must be cautious inferring a 
causal connection between the GDE transformation and student outcomes given limitations 
in the data and research design (discussed above). Additionally, our analysis was not 
designed to determine why GDE students outperformed non-GDE students. To improve 
academic outcomes for both Green Dot and other public school students, it is important to 
identify the specific educational mechanisms responsible for the observed positive results. 
Doing so will require much more in-depth documentation and analysis of the people, 
programs, and policies that distinguish Green Dot Public Schools from more traditional 
public schools. These more intensive research efforts are necessary, however, to identify 
effective and actionable avenues for educational reform. 

Nevertheless, the Green Dot Public School’s transformation of Alain Eeroy Eocke High 
School has been a success for the majority of students in the transformation’s first two 
cohorts. It is also encouraging that GDE accomplished positive effects on student 
achievement while maintaining a student population similar to its original population prior to 
transformation and to the control schools used in the study. Additionally, the increasingly 
positive results for Cohort 2 students, relative to Cohort 1 students, suggests positive benefits 
are likely to materialize for successive cohorts as well. As GDE’s story progresses, future 
chapters on additional cohorts of students may further solidify the evidence base. 


22 



References 


Herman, J. L., Wang, J., Rickies, J., Hsu, V., Monroe, S., Leon, S., Staubhaar, R. (2012). 

Evaluation of Green Dot’s Locke Transformation Project: Findings for Cohort 1 and 2 
Students. Los Angeles: University of California, National Center for Research on 
Evaluation, Standards, and Student Testing (CRESST). 

Ho, D., Imai, K., King, G., & Stuart, E. (2009). Matchit: Nonparametric Preprocessing for 
Parametric Causal Inference. Retrieved from: 
http://gking.harvard.edu/matchit/docs/matchit.pdf 


23 




Appendix 


Classification of key college-prep courses based on Green Dot and LAUSD course titles 


Subject Classification 

LAUSD Course Titles 

Green Dot Course Titles 

Key English Courses: Year 1 

ENGLISH 9A 
ENGLISH 9B 
ESL ADV 3 
ESL ADV 4 

English 9A 
English 9B 
English Honors 9A 
English Honors 9B 
ESL Adv. 3/4 A 
ESL Adv. 3/4 B 

Key English Courses: Year 2 

ENGLISH lOA 
ENGLISH lOB 

English lOA 
English lOB 
English Honors lOA 
English Honors lOB 

Key English Courses: Year 3 

AM LIT COMP 
AP ENG LANG A 
AP ENG LANG B 
AUTH COMP A 
AUTH COMP B 
CONTEMP COMP 

AP English Language A 
AP English Language B 
English 11 A: American Literature 
English IIB: American Literature 
English Honors 11 A: American 
L. 

English Honors IIB: American 
L. 

Key English Courses: Year 4 

AP ENG LIT A 
AP ENG LIT B 
ENGLISH LIT 
EXPOS COMP 
MODERN LIT 
WORLD LIT 

AP English Literature A 
AP English Literature B 
English 12A 
English 12B 
English Honors 12A 
English Honors 12B 

Key Math Courses: Year 1 

AEG lA-LAVA 
AEG IB - LAVA 
ALGEBRA lA 
ALGEBRA IB 

Algebra lA 
Algebra IB 

Key Math Courses: Year 2 

GEOM A - LAVA 
GEOM B - LAVA 
GEOMETRY A 
GEOMETRY B 

Geometry A 
Geometry B 
Honors Geometry A 
Honors Geometry B 
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Subject Classification 

LAUSD Course Titles 

Green Dot Course Titles 

Key Math Courses: Year 3 

ALGEBRA 2A 

Algebra 2 Honors A 


ALGEBRA 2B 

Algebra 2 Honors B 
Algebra 2A 
Algebra 2B 

Key Math Courses: Year 4 

AP CALC A UCCP 

AP Calculus A 


AP CALC A-LAVA 

AP Calculus B 


AP CALC B UCCP 

Calculus A 


AP CALC B-LAVA 

Calculus B 


AP CALCULUS A 

Math Analysis A 


AP CALCULUS B 

Math Analysis B 


AP CALCULUS C 

Pre Calculus A 


AP SEAT A UCCP 

Pre Calculus B 


AP SEAT B UCCP 

Trigonometry A 


AP SEAEISEICS A 

Trigonometry B 


AP SEAEISEICS B 

Trigonometry Honors A 


DISCR MAEH A 
DISCR MAEH B 
MAEH ANALY A 
MAEH ANALY B 
SEAL & PROB A 
SEAE & PROB B 
ERG/MAEH AN A 
ERB/MAEH AN B 

Trigonometry Honors B 

Key Science Courses: Year 1 

BIOLOGY A 

Biology A 


BIOLOGY B 

Biology B 


ADV BIO A 

AP Biology A 


ADV BIO B 
AP BIO A 
AP BIO A-LAVA 
AP BIO B 
AP BIO B-LAVA 

AP Biology B 

Key Science Courses: Year 2 

CHEMISERY A 

Chemistry A 


CHEMISERY B 

Chemistry B 


AP CHEMISERY A 

Chemistry Honors A 


AP CHEMISERY B 

Chemistry Honors B 
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Subject Classification 

LAUSD Course Titles 

Green Dot Course Titles 

Key Science Courses: Year 3 

PHYSICS A 
PHYSICS B 
AP PHYSICS A 
AP PHYSICS B 

Physics A 
Physics B 
Physics Honors A 
Physics Honors B 

Key History/Social Science 
Courses: Year 1 

WHO: MOD WLD A 
WHO: MOD WLD B 
AP WLD HIST A 
AP WLD HIST B 
AP EUR HIS A 
AP EUR HIS B 

World History A 
World History B 
World History Honors A 
World History Honors B 
AP World History A 
AP World History B 

Key History/Social Science 
Courses: Year 2 

US HIST 20TH A 
US HIST 20TH B 
AP US HIST A 
AP US HIST B 

U.S. History A 
U.S. History B 
U.S. History Honors 11 A 
U.S. History Honors 11 B 
AP US History A 
AP US History B 
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